Information systems increasingly leverage artificial intelligence (AI) and machine learning (ML) to generate value from vast amounts of data. However, ML models are imperfect and can generate incorrect classifications. Hence, human-in-the-loop (HITL) extensions to ML models add a human review for instances that are difficult to classify. This study argues that continuously relying on human experts to handle difficult model classifications leads to a strong increase in human effort, which strains limited resources. To address this issue, we propose a hybrid system that creates artificial experts that learn to classify data instances from unknown classes previously reviewed by human experts. Our hybrid system assesses which artificial expert is suitable for classifying an instance from an unknown class and automatically assigns it. Over time, this reduces human effort and increases the efficiency of the system. Our experiments demonstrate that our approach outperforms traditional HITL systems for several benchmarks on image classification.

If you've been keeping up with us, you know that our core goal is to deliver extreme agility with no limits, allowing everyone to innovate. And if you're new here, now you know what OutSystems is all about. A future with no limits is one where intelligent tools augment the work of users with the expertise learned from millions of anonymized code patterns. At outsystems.ai, we're building that future with the ultimate goal of making app development 100x faster, while enabling pro developers and business people to deliver robust, high-quality applications with different levels of complexity. That's why we're happy to announce the general availability of our next-generation AI-Assisted Development capability, which is built right into Service Studio -- the OutSystems development environment.